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ABSTRACT 

The Systems Engineering Simulator (SES) 
houses a variety of real-time computer 
generated visual systems. The earliest 
machine dates from the mid-1960's and is 
one of the first real-time graphics 
systems in the world. The latest 
acquisition is the state-of-the-art 
Evans and Sutherland CT6 . Between the 
span of time from the mid-1960's to the 
late 1980's, tremendous strides have 
been made in the real-time graphics 
world. These strides include advances 
in both software and hardware 
engineering . 

The purpose of this paper is to explore 
the history of the development of these 
real-time computer generated image 
systems from the first machine to the 
present. Hardware advances as well as 
software algorithm changes are 

presented. This history is not only 
quite interesting but also provides us 
with a perspective with which we can 
look backward and forward. 

ACRONYMS AND ABBREVIATIONS 

CIG Computer Image Generator 

CT3 Continuous Tone Computer Image 

Generator - third generation 

CT6 Continuous Tone Computer Image 

Generator - sixth generation 

EPU Edge Processing Unit 

ESG Electronic Scene Generator 

JSC Johnson Space Center 

MMU Manned Maneuvering Unit 

NASA — National Aeronautics and Space 
Administration 

OGU Object Generating Unit 

OMV Orbital Maneuvering Vehicle 

R520 — Raytheon 520 

SAIL — Shuttle Avionics Integration 
Laboratory 

SEL Systems Engineering Laboratories 

SES Systems Engineering Simulator 

SGS Surface Generator Subsystem 

TOU Timing and Output Unit 

VCU Vector Calculating Unit 


INTRODUCTION 

In the twenty odd years between the 
first real-time computer image generator 
to the present, many strides have taken 
place to provide realistic, full color, 
three-dimensional displays for use in 
many areas in the simulation community. 
NASA/JSC is rare in that it contains a 
snap-shot of this development 
approximately every ten years. 

From the first of its kind to the 
current, real-time computer image 
generators provide the necessary visual 
displays to support the increasingly 
heavy demands placed on the Systems 
Engineering Simulator (SES). This paper 
explores the history of the scene 
generators which existed and still exist 
in the SES. 

The hardware configuration and new 
technology of each graphics system is 
explained. The salient features and 
innovations of each system as they were 
introduced to the SES is explored. 
Several advances in the theory of 
database modeling have evolved 
throughout the years and real-time 
programming has changed from minimal to 
extensive . 

Due to historical carryover, the terms 
Electronic Scene Generator (ESG) and 
Computer Image Generator (CIG) will be 
used interchangeably. 

SES — SYSTEMS ENGINEERING SIMULATOR 

The Systems Engineering Simulator (SES), 
formerly the Shuttle Engineering 
Simulator, has been in continuous 
operation since the programs conception 
in 1968. The SES supports real-time 
man-in-the-loop computerized engineering 
simulation for the Shuttle, space 
station, and other space related 
programs and projects. 
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The two main areas of operation 
utilizing Electronic Scene Generators 
(ESG) are entry and on-orbit. 

The entry simulation is hosted by a 
Cyber 840. The orbiter forward cockpit 
mockup is located in the East High Bay 
of Building 16. 

On-orbit simulation is accomplished with 
the use of five SEL 32/8780 supermini 
digital computers and four SEL 32/75 
digital computers. Mockups include an 
orbiter aft station, MMU station, and a 
cupola station. The cupola is the 
operations station for the space 
station. All of these mockups reside in 
Building 16. On-orbit operations which 
are supported include space station 
docking/berthing, payload 
handling/deployment, MMU operations, and 
OMV studies to mention a few. 

NASA I — THE ORIGINAL SCENE GENERATOR 

"And, in the beginning, there 

was texture . " 

NASA/JSC was instrumental in bringing to 
fruition the concept of real-time 
computer generated images. In the time 
before the mid-1960's, the 
out- the-window visual images were 
generated by model boards: large, 
scaled replicas of the simulation 
terrain over which a closed circuit 
television camera traversed. These 
model boards were built specifically for 
the purpose at hand and not easily 
modifiable . 

A new concept emerged in the early 
1960's. Although rather idealized, the 
scenes produced by computers could be 
generated in real time to satisfy the 
requirements to provide scenes for 
out-the-window displays. In August, 
1964, NASA at the Manned Space Center 
installed the first such computer 
device. The dawn of real-time computer 
generated images began with the "Visual 
Contact Analog: Three-View Interim 
Space-Flight Simulator" build by General 
Electric . 

As intimated above, this computer system 
produced three views. These views 
consisted of an unbounded textured 
planar surface for the ground. This 
special purpose computer, the Surface 
Generator System, calculated the 
perspective transformation of a surface 
texture. The optical system displayed 
the resulting pictures so that the 
environment appeared distant to the 
observer . 

There were a few interesting details to 
be discovered about this system. Due to 
the state of the digital art at that 
time, several problems were solved with 
analog methods. The textured surface 


was computed in digital form without a 
roll angle. This made the algorithm 
simpler and roll was accomplished in the 
circular television monitor by 
electronically rolling the raster. 
Precise nonlinear sweeps were generated 
by the display unit to compensate for 
optical distortions. To avoid 
disturbing moire patterns, fine detail 
was gradually faded out of the picture 
with analog circuitry. 

The entire computer system consisted of 
six pipelined processors built with 
pre-TTL equipment: Computer Control 
Corporation (3C) circuitry cards 
containing discrete components. 

Screaming along at 5 MHz, the displays 
were generated at 30 frames a second. 
This corresponds to the current American 
commercial standard. 

As in all computer image generators, the 
first processor unit is the unit with 
the highest programmability. The first 
unit of the Surface Generator System, 
the Program Control Unit, contained 512 
48-bit words and had a memory access 
time of 5 microseconds. 

This machine served the Guidance and 
Control Division for several years. 


NASA II -- THREE DIMENSIONAL CAPABILITY 

In February, 1968, modifications were 
made to the Interim Visual Space Flight 
Simulator and a large complement of 
equipment was added. 

An innovation occurred in the field of 
computer generated images. 
Three-dimensional objects were added to 
the textured surface. In this time 
frame, the Manned Space Center was 
heavily involved with moon landings. 
The additional capability provided the 
simulation with idealized forms of lunar 
mountains and craters as well as the 
traditional realistic landing fields. 

The new system consisted of the 
following components: 

1. A Raytheon 520 ( R 5 2 0 ) general 
purpose computer. Flexibility was 
introduced by linking a general purpose 
computer to a set of special purpose 
computers. Although minimal by modern 
standards, the memory capacity was 8096 
24-bit words core memory and 256 24-bit 
words of high speed memory. This 
concept formed the basis of flexibility 
in the succeeding generations of 
real-time visual systems. 

2. A Vector Calculating Unit (VCU). 
This special purpose computer deviated 
from the classical Von Neumann computer 
architecture. it contained a 4096 


220 



24-bit word program memory unit and 
three sets of 2048 30-bit word data 
memory units corresponding to the X, Y, 
and Z components of the Cartesian 
coordinate system. The data memory 
units can be accessed in parallel, 
enabling dot and cross product 
calculations to be made quickly. 

3. A set to two Object Generating Units 
(OGU). Each unit was capable of working 
on twenty objects made up of eighty 
faces and described by a maximum of 120 
edges. For each edge of the 
environment, one circuit board was 
requi red . 

4. A Timing and Output Unit ( TOU ) . 
This unit performed three functions. It 
generated the master timing signals for 
the entire system. The TOU served as a 
mixing and distribution point for the 
video outputs from the two OGU's and 
routed data from the VCU to the OGU's, 
SGS , and displays. Test patterns were 
generated in the TOU for aligning and 
trouble shooting the system. 

The Surface Generating Subsystem (which 
was the Visual Three-View Space-Flight 
Simulator) was modified to allow its 
operations at 20 frames a second. The 
NASA II system operated at a slower rate 
due to the constraints imposed on it by 
the R520 and VCU. 

Of interest here is the fact that the 
objects which were generated came from a 
catalog of two-dimensional polygons and 
three-dimensional objects. Each OGU had 
the capability of generating one 
decahedron, one octahedron, two 
hexahedrons, and four tetrahedrons as 
well as two dodecagons, three octagons, 
four hexagons, three quadrilaterals, and 
two triangles. The maximum capability 
of 120 edges per OGU could not be 
exceeded. The combined capacity of the 
two OGU's was 240 edges. 

The database designer had to create a 
scene choosing objects and polygons from 
a catalog of available objects and 
polygons. The specification of vertices 
for the objects and polygons followed 
stringent rules. The concepts of 
planarity and convexity were required 
for each planar surface. Because object 
topology was predefined, vertex 
selection required a lot of 
precalculation for irregular objects. 
Selection of the objects and polygons 
amounted to filling specific absolute 
locations in the R520 memory. Color 
selection followed a similar procedure. 

The designer also had the choice of one 
quadrilateral shadow polygon and one 
beacon. The shadow polygon emulated the 
shadow created by ones ownship. It 
changed configuration in response of the 
vehicles attitude with respect to the 


surface and an imaginery sun. The one 
beacon was a two element by one line 
pair dot. It had the capability of 
flashing and the period was 
programmable . 

The real-time programming consisted 
mainly of calling the subroutines which 
transformed objects and polygons in the 
correct order. At this point in time, 
there was not a clear distinction 
between database design and real-time 
programming. The two concepts were 
closely intertwined. 

The NASA II system, therefore, presented 
an environment consisting of 
three-dimensional objects on a two 
dimensional textured surface. 

NASA III — A BETTER WAY 

In November, 1971, a major innovation 
was incorporated into the then current 
visual system. The two OGU's with its 
combined capacity of 240 edges was 
replaced with an Edge Processing Unit 
(EPU) which increased the edge 
capability to 320 edges. The theory of 
a fixed set of objects and polygons was 
superseded with a more general approach 
of just polygons. Groups of polygons 
were gathered to form three-dimensional 
objects . 

A new concept was also introduced. 
Edges could now be shared between an 
object or among objects which did not 
move relative to each other. This 
provided an addition edge capacity 
capability. For example, a cube has six 
sides and each side has four edges. 
With this method, a cube could be 
described as six sides with twelve 
shared edges rather than six sides with 
four edges each for a total of 24. 

With the added flexibility that this new 
system brought, a need for programs to 
generate databases was required. The 
first database compiler was written by 
Lockheed at the NASA Manned Spacecraft 
Center. Because the program was written 
in an early version of Fortran, the 
syntax was necessarily field sensitive. 
Things had to be in the right column. 

The database designer specified clusters 
by grouping polygons. He specified 
polygons by grouping vertices. Clusters 
had to follow some rigid constraints. 
NASA III used the idea of separating 
planes. Clusters were separated from 
each other by invisible, infinite planes 
called nodes. Modules were groups of 
clusters which did not move relative to 
each other. Each polygon was given an 
attribute such as color, back-face 
generation, and shadow or beacon 
generation. Given the capacity of the 
machine, the number of edges per polygon 
was completely arbitrary. One of the 
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test patterns which existed on the 
machine has sixteen polygons of twenty 
edges each. 

The real-time program, which resided in 
the R520 and the VCU, was written by 
General Electric. It was an upgraded 
version of the one which resided in the 
R520 in NASA II but the new algorithm 
made the program much less complex and 
easier to manage. 

Programming of the real-time software 
was rather trivial. The program was 
mainly driven by the database 
environment. It required little 

modification for each new environment. 
The environment consisted mainly of two 
independent coordinate systems which 
could contain an eyepoint or modules, 
one coordinate system which could 
contain only an eyepoint, and the ever 
present textured surface. 

With the advent of the CT3, described 
below, the NASA III system was renamed 
the Electronic Scene Generator #1 (ESG 
# 1 ). 

CT3 — A MAJOR STEP 

A major step in the evolution of 
real-time computer generated image 
systems was made in November, 1976. The 
CT3 made by Evans and Sutherland was 
introduced to the SES. This system had 
many new attributes which deserve 
mentioning. This system is currently 
employed to a great extent in the SES 
laboratory . 

The CT3 consists of three general 
purpose computers of the PDP-11 series, 
a visual pipeline, and a collision 
detection pipeline. 

A central PDP-11/40, called the HOST, is 
interfaced to the simulation laboratory. 
The HOST interfaces to the visual system 
and the collision detection system. The 
main purpose of this machine is to 
gather the data from the simulation, 
format the data, and send the results to 
the other subsystems. 

The visual subsystem is driven by a 
It is connected to a visual 
pipeline containing 10 programmable 
special purpose processors. Two 
independent channels of visual images 
are produced. The total capacity of the 
visual system is 900 polygons. 

A separate collision detection system 
allowed the simulation to detect the 
intersection of impenetrable objects. 
This system consisted of a PDP-11/45 as 
well as a collision detection pipeline 
containing two programmable special 
purpose processors. 

The frame rate of this system was 25 Hz. 


This corresponds to the European 
commercial standard. Although ESG #1 
was operating at 20 Hz, no problem was 
presented. CT3 was used for on-orbit 
studies and ESG #1 was not. 

The visual system included many new 
features which are described below. 

Anti-aliasing and edge smoothing were 
added to improve picture quality. 
Spatial filtering was used as the 
algorithm. 

Directional illumination was introduced 
to provide an illusion of sun direction, 
intensity, and environmental depth. 

Smooth-shaded polygons simulated round 
or complex shaped objects. The Gouraud 
shading algorithm was implemented in 
hardware using extremely fast ECL 
circuitry. 

Hidden surface removal by range priority 
was done in hardware as well. This 
eliminated the necessity for separatinq 
planes. 

A separate modeling system was delivered 
which was used to create and analyze 
databases. This system consists of a 
calligraphic display system and a 
general purpose computer, PDP-11/40. 
Software packages aid the designer in 
producing new databases. The databases 
are viewed in wireframe on the display 
system. 

This is the first time that the database 
modeling was implemented on a system 
other than the visual system. Due to 
the extremely heavy usage of the CT3 in 
the simulation laboratory, a separate 
modeling station is necessary. 

A new language was developed for 
database modeling. The new compiler 
set, MEDUSA for the visual system and 
COLIDE for the collision detection 
system, offers the designer a capability 
and flexibility heretofore unknown. In 
the process of constructing an 
environment, the designer defines and 
names points. Subsequently, he defines 
polygons in terms of these points and 
adds attributes such as color and 
reflectivity. With this innovation, the 
database design becomes much more like 
programming rather than filling out 
specification requirements. 

Another language set, VIS for the visual 
system, CDS for the collision detection 
system, and HOST for the host machine, 
was developed for the real-time 
programmer. Symbolic parameter areas 
are defined by the programmer and 
transformation sequences are specified 
to meet the simulation requirements. 
Again, the real-time programmer is more 
of a programmer rather than someone who 
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allows the data to dictate the real-time 
configuration. Currently, two versions 
of the HOST software exists. HSTSES 
interfaces to SES and HSTSTS interfaces 
to SAIL. At the present time, CT3 is in 
use at least 16 hours a day supporting 
on-orbit studies in the SES. 

ESG #1 — A NEW LEASE ON LIFE 

Returning to ESG #1 (alias NASA III), 
the Raytheon 520 was more and more 
difficult to maintain and the SGS was 
even more difficult to maintain. During 
the early 1980's, this author's project 
was to replace the R520 with a more 
modern Systems Electronic Laboratory 
(SEL) 32/55. During this project, the 
SGS was removed from the system as well 
as some other parts of old ESG #1. The 
remaining units are the VCU, TOU , and 
EPU. The display systems were reworked. 
This effectively removed the texture 
capability from the system. 

The frame update rate of ESG #1 was not 
changed. To have done so would have 
required major hardware modifications. 
Therefore, the frame update rate was 
retained at 20 Hz. 

Along with the hardware changes, a new 
set of software packages had to be 
written. Rather than follow the 
original design for database modeling 
and real-time application software, it 
was desirable to model the software 
after CT3 . In this effort, the database 
modeling software closely approximates 
that of CT3 , given the different 
hardware algorithms between the two 
machines. in addition, the real-time 
software approximates the real-time 
software of CT3 . The desired effect was 
the capability of a database designer or 
real-time programmer to move between the 
two machines with very little effort. 

The database modeling software, ENCOM, 
was designed to emulate the database 
modeling software which exists on CT3. 
A complete rework was undertaken so that 
a programmer on CT3 could move to ESG #1 
with minimal effort. The established 
procedure of defining and naming points 
followed by defining polygons in terms 
of these points was introduced to ESG 
# 1 . 

The real-time software followed a 
similar theory and procedure. The 
real-time software on CT3 was deemed a 
standard and the real-time software on 
ESG #1 was designed to match as closely 
as possible. Currently, three versions 
exist. VISUAL interfaces to the SES, 
VISSTS interfaces to SAIL, and VIS is a 
standalone version used for local 
applications and development. 

As of this writing, ESG #1 is actively 
supporting entry simulation. A set of 


13 entry scenes are available. Due to 
its age, it is difficult to maintain and 
some of the electronic components are 
not obtainable. There is a project 
underway to retire it and replace it 
with a more modern computer image 
generator . 

POLY 2000 — A MEDIUM RESOLUTION, LOW 
COST APPROACH 

A POLY 2000 built by GTI , Incorporated 
( GTI ) was purchased in October, 1985. 
It was planned to add three low 
resolution channels to the complement 
which existed in SES at that time. 
However, the software and hardware 
theory was radically different. 

The digital technology had advanced so 
greatly during this period in time that 
the POLY 2000 could accomplish real-time 
computer generated images using 
micro-coded, high-speed bit slice 
processors . 

The POLY 2000 at that time was in its 
infancy. It did not have anti-aliasing 
or smooth shading. It did have diffuse 
reflectivity which gave the impression 
of a sun direction. Alphanumeric 
characters could be overlaid on the 
scene . 

The POLY 2000 consisted of a general 
purpose computer and a set of special 
purpose bit-slice processors. The 
general purpose computer, the Alcyon, 
was used in database development as well 
as generating load modules for the POLY 
2000 proper. The first major processor 
was the System Control Module (SCM). 
When the system was connected to the 
simulation, data was sent directly to 
the SCM and the Alcyon was not used. 
This left the Alcyon free to be used in 
other capacities. 

Special requirements were insisted upon. 
The frame update rate was set at 25 Hz 
to match that of CT3 . Multiple channels 
were requited. The vendor produced a 
system with three independent channels. 

Database modeling was done on the Alcyon 
and the binary object files were stored 
on its disk. When needed, the object 
modules were downloaded to the SCM. The 
database compilers, polyi2l and polyd2D, 
were not compatible with any of the 
database modeling software which existed 
in the SES. 

The real-time programs were written in C 
on the Alcyon and, also, downloaded when 
required. No special language was 
implemented; all special functions were 
included in a library. 

These concepts were radically different 
than the ones established on ESG #1 and 
CT3 . Lockheed personnel undertook the 
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effort to write a set of software 
packages which would more closely 
emulate the concepts of the other 
graphics systems. These included the 
database compiler (POLYS) and the 
real-time support software packages 
known as the linker (LOADER), and the 
loader (LOAD). The syntax of the 
language was modeled after those 
commands found in the software of the 
other systems. 

POLY 2000e — AN ENHANCED VERSION 

Given a few years, GTI enhanced the POLY 
2000 to include additional features. 
Their new POLY 2000e provided the 
following enhancements: 

1. Smoo th- shading - Gouraud 
shading was added to the flat shaded and 
fixed shaded polygons already in the 
system. 

2. Anti-aliasing - Sub-pixel 
averaging was used as the algorithm for 
curing the jaggies. 

3. Transparency - Eight levels 
of polygon transparency was incorporated 
into the system. 

4. Depth fogging - Gradual 
dissolution to the background provided 
for fog and haze. 

Essentially, the enhanced system was an 
entirely new system. New algorithms and 
hardware were exchanged for the older 
system. The three channels operating at 
25 Hz were retained. The general 
purpose computer, the Alcyon, was also 
retained . 

Because the hardware and software 
theories were changed, the custom-made 
compilers had to be changed. Extensive 
effort was employed to upgrade the 
compilers to match the new system 
without having to rework all the 
previous database and applications 
software. 

The three channels of the POLY 2000e are 
currently supporting on-orbit studies in 
the SES. 

CT6 — A PRIDE AND JOY 

As of this writing, an Evans and 
Sutherland CT6 is in the process of 
being integrated into the SES 
laboratory. As with the advent of CT3, 
major steps in computer graphics were 
introduced to the SES. 

The general purpose computer of the CT6 
is a Gould Concept 32/67. The special 
purpose computers are still arranged in 
a pipeline fashion but there are fewer 
major components doing much more. The 
configuration of the pipeline allows 


channelization to be performed early in 
the processing. 

The current system has six independent 
channels of high resolution, high 
quality images. The system is capable 
of processing thousands of polygons and 
each channel can display up to 1500 
polygons. Compared to the hundreds in 
the other systems, the increased 
capacity is impressive. 

Not all the database need reside in 
active memory. Parts of the database 
which are potentially visible reside in 
memory while the rest resides on disk. 
When those objects not in memory become 
potentially visible, they are paged in 
from disk to memory. This provides a 
mechanism which essentially expands the 
potentially visible database many fold. 

Texture returns. Not only ground 
texture but polygonal texture is made 
available. Any polygon regardless of 
orientation can have texture. Many 
texture patterns are available in the 
system. Texture can be produced by 
closed form equations or 
photographically derived. 

Database development is now necessarily 
more complex. Database modeling is done 
on a MicroVax II. Not just one software 
package is enough. Several database 
modeling software packages are provided 
not only to develop databases but also 
to display the resulting databases on a 
color calligraphic display device, the 
PS330. Database development includes 
object and surface production 
capability. The main database compiler, 
DBC , deals with objects, polygons, and 
points. A separate linker, LNK , is used 
to join the intermediate binary object 
files. The surface feature editor, SFE, 
assists in creating large terrains 
principally used in landing scenarios. 
Several display packages are included 
such as the CT simulator, CTS, and the 
graphics editor, GRE . 

Complexity also manifests itself in the 
real-time software. The increased 
memory and speed provided by the Gould 
Concept 32/67 also provides a wealth of 
capabilities for the real-time system. 
Because the Gould Concept 32/67 is a 
dual CPU system, the real-time tasks 
have been divided to optimize this 
feature. The user interface to the 
real-time process, RTS, allows the user 
a wide variety of commands and 
capability to control and configure the 
system. 

As of this writing, the CT6 is in the 
last stages of integration into the SES. 
The database designers are working hard 
to supplement the databases already 
delivered by Evans and Sutherland. The 
SES is looking forward to the time when 
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the CT6 will be brought on-line and 
support the simulation studies. 


CONCLUSION 


This paper has explored the history of 
the computer image generator as they 
existed and still exist in the SES. 


Many advances have 
hardware, taking 
current 
available 
discrete 
integrated 


been made in the 
advantage of the 
state-of-the-art circuitry 

at the time. From the 
components to the first 
circuits to the very large 
scale integrated devices, the real-time 
graphics industry has tried to use 
everything at its disposal to create the 
best images available. The theory on 
how to best utilize the hardware 
advances has also changed toward 
flexibility, programmability, and 

manageability. 

Database design has grown from data 
specifications to large and complex 
programs. As the complexity in 
databases increased, the complexity in 
the database software increased. 


Real-time software has changed radically 
over the duration of the real-time 
computer image generator. In the early 
stages, the simulation visual system was 
driven mainly by the data which it 
received. As more and more powerful 
front-end general purpose computers were 
available, the real-time programmer was 
able to enjoy more and more flexibility 
in the control of the visual system. 
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FIGURE 1 : BLOCK DIAGRAM OF NASA II 



FIGURE 2: BLOCK DIAGRAM OF NASA III 











FIGURE 3: BLOCK DIAGRAM OF CT3 



FIGURE 4: BLOCK DIAGRAM OF NASA III, UPGRADE 
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FIGURE 5: BLOCK DIAGRAM FOR POLY 2000 AND POLY 2000e 



FIGURE 6: BLOCK DIAGRAM FOR CT6 
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